Skip to content

feat: the VDK, Visor Development Kit - #91

Merged
macedonga merged 88 commits into
mainfrom
kit
Nov 9, 2026
Merged

macedonga merged 88 commits into
mainfrom
kit

Conversation

@macedonga

@macedonga macedonga commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Visor's side of the VDK, the Visor Development Kit. Apps on the phone, web apps on the Display, programs and AI
agents on a computer, and the visor command over adb use the glasses through Visor, with the person's consent per
app and per scope. Visor 3.0.0 is the first Visor that serves it.

The kit comes from Maven Central, com.macedonga.visor:kit 0.1.1. With ../visor-kit checked out next to Visor,
the build takes it from there instead. Docs: https://visor.macedonga.com/developers/

In Visor

  • Settings > Apps, under Glasses: the phone apps with access, each with a sheet that takes back a scope or all of
    them; the Display's web apps with their access, adding one or one from Meta's catalogue, and their tools for the
    assistant; programs and agents on the network door with their own tokens; Developer mode and its token.
  • A consent screen per app and scope, with "only this time".
  • The kit service for apps (AIDL), the adb socket for the visor command, and a network door on 127.0.0.1:7333 with
    JSON-RPC over WebSocket, MCP and webhooks (local network opt-in, one token per program).

What apps get

  • The glasses: status with the thermal level, pairs and pair select, touch, buttons and the Display's navigation as
    inputs, motion with orientation fused on the phone.
  • The camera: photos, videos and captures, and the Live view as a feed (camera.live, camera.live.photo).
  • The Display: layouts in the toolkit app's own format, videos, the teleprompter, notifications, and app cards on the
    Briefing panel with details, pages and charts.
  • The Neural Band: status, buzz, wrist, gestures and motion, shared with the Display through the phone.
  • Voice and the assistant: transcripts from the glasses' microphone and from the Display's speech, voice.speak,
    voice phrases that start an app, questions routed to apps, apps' tools offered to the assistant, and the
    assistant's turns and history.
  • The glasses' settings (settings.*): every setting readable, the everyday ones writable. Updates, resets, heat
    protection, developer and logging switches stay read-only, and passwords are never shown.
  • Web apps on the Display: every method from JavaScript, the phone's sensors, location check and Bluetooth scans,
    and their WebMCP tools for the wearer's own model.
  • Raw service messages, behind Developer mode only.

Apps bind the kit service and get what the person grants them on the
consent screen; Settings lists them and takes access back. While
Developer mode is on, the visor command reaches the same methods
through the adb socket. Band lessons are counted, so the gestures
screen and kit apps can hold the band together, and the photo switch
stays on while an app holds the camera scope.
…ost apps' notifications

A question that holds a VDK app's phrase goes to that app instead of
the model, and its reply is spoken like an answer; without a reply in
time, the wearer hears that the app did not answer. Apps' tools join
the model's toolbox. Apps' notifications show on the Display.
Off by default. When on, Visor serves the kit on 127.0.0.1:7333, or on
the local network too: JSON-RPC at /rpc and MCP at /mcp. Each client is
one the person made in Advanced settings, with its own scopes and a
token shown once and kept only as a hash.
Identity, the link's authentication and firmware updates stay out of
reach.
Pairs come from Visor's own store, last-seen record, Meta's looks and
the bands; the store is read on demand after a fresh start. Selecting
a pair switches as the home's pick does, and refuses during a Live or
an import.
A client may get a webhook URL and topics; Visor posts each event as
JSON, signed with HMAC-SHA256 keyed by the hash of the client's token.
A wrong token is now turned away with 401 before the WebSocket opens,
and glasses.status waits until Visor has read its store, which before
read as no glasses set up.
The first app to watch starts the Live unless it runs already, and the
last one stops it if an app started it. Each watcher gets the codec
configs first and video from a keyframe; after lost frames it waits for
the next keyframe and asks the glasses for one. A live photo reaches an
app as a FileProvider URI it may read, and a computer as Base64.
Each push is read on its own, over Bluetooth or the Live's Wi-Fi
session, since the merged device state would hide a second tap like the
first. The opening snapshot repeats old presses, so it is skipped. The
consent screen names the new inputs scope.
One headset recording and one recogniser session serve every app that
listens, in one language. headsetListen reads the glasses' call mic
until cancelled and never passes on audio routed anywhere else; the
recogniser is GlassesEars, fed PCM instead of the glasses' Opus.
The band paired to the Display by Meta's app has no record in Visor, so
band.status read it as unpaired. It now falls back to the band battery
in the Display's device state, as Visor's own screens do, with held
false until the band is set up in Visor.
Apps read every setting and change the everyday ones; updates, resets, heat protection, developer and logging switches and encoded settings stay read-only, and passwords are never shown.
…tant's diary

With the conversations scope, apps get each question and answer as it is answered, and the history kept on the phone, newest first.
…n svc6500

Session start and stop in developer mode, capability requests split at 15000 bytes as the SDK splits them, responses and events, and the IMU's start, stop and samples. Checked against the DAT SDK 1.0.1's own message classes.
Columns, rows, text, images, buttons, cards and clickable boxes become DAT's own components with its sizes, fonts and colours, and clicks come back as the node's id. Checked against the DAT SDK 1.0.1's Display renderer. The codec gains the display content, clear, click and back messages.
… session on it

The link proves an app key on svc79 against the constellation manifest, as DAT
apps do; RLDrive grants the toolkit permissions. The debug probe got motion,
a layout and its clicks from the Display's toolkit app.
One StreamControlReq turns the gestures and the IMU streams (IMU_GYRO 15,
IMU_ACCEL 16, IMU_MAG 17, IMU_QUAT 18) on together, so neither switches the
other off. Samples merge the newest reading of each sensor and go out at the
rate asked for. The motion consent now names the band, in all four languages.
…eal BLE scans

svc41120 serves the phone's accelerometer, gyroscope, magnetometer, linear
acceleration, gravity and orientation to the Generic Sensor API with sensorId
phone. svc51 answers the browser's user location check before geolocation.
BLE scans on svc55074 now scan for real, with the page's filters on the phone
and in hardware where Android has one. Settings > Web apps lets each app scan
for nearby Bluetooth devices, which writes the Display's origin allowlist,
web_on_hn.webapp_ble_scan_allowed_origins.
Sharing now says where it is on the band settings row: connecting, waiting
for the glasses, or on once the Display and the band talk through the
phone. When the phone cannot reach the band in 60 s, the two never start
their link through the relay in 30 s, or that link keeps dropping, sharing
turns itself off, the band goes back to the Display and the row says why.
band.motion ends with not_shared when sharing stops, and names the setting.
KitWebApps turns visor-kit's scan frames into JSON-RPC sessions with the
scopes the person gives each web app's origin; a real scan still scans.
Settings > Web apps opens a sheet per app: Visor access by scope, which
keeps Nearby Bluetooth devices on with it, saved to the glasses in one go
with a line that says when the first read of their configuration takes a
minute. Removing an app takes its access away.
Settings > Web apps turns on web_on_hn.webapp_webmcp and
wearables_agentic_nova.forward_webmcp_requests_to_server, so a turn with a
web app's tools comes to the phone with those tools on ev57 field 54. Tools
Visor does not know get the glasses' own description and schema from an MCP
tools/list on ev311, asked once a link with a 1.5 s limit, and keep the
generic help when the glasses do not answer. Both config writers now say
when the glasses' software lacks the flag instead of skipping it.
@macedonga macedonga added the enhancement New feature or request label Nov 9, 2026
@macedonga macedonga added this to the 3.0.0 milestone Nov 9, 2026
@macedonga macedonga self-assigned this Nov 9, 2026
Rows, section labels and notes straight on a sheet used the card's 16dp
and the page's 32dp, so a sheet had three left edges. On a sheet they now
take SIDE, as the title and the answers do; on a card nothing changes.
Settings gets Apps under Glasses: the phone apps with access, the
Display's web apps, programs and agents on the network door, and
Developer mode. Each phone app opens a sheet that takes back a scope or
all of them. The web apps move in from their own page, so they have one
place: open, add, browse Meta's, and their tools for the assistant.
Advanced keeps the rest. The scope lists show the scopes' names, and the
README gets a section on apps for the glasses.
A COMMAND frame shares DataX's bit 7, so the reader measured a relay's
commands as DataX: the long ADVERTISEMENT 81 80 read as 388 bytes, the
DATA frames after it were swallowed, and the link dropped on the next
byte. Sharing the band with the Display looped on that. Bare DataX only
crosses in the handshake, so once both its messages are in, the reader
takes bit 7 as a COMMAND: 2 bytes, or 5 + length in its long form.
Opening a web app from the phone does not work on the Display: the
browser's service registers svc55074 only on a connection that closes
once Android freezes it, and the glasses answer every open with
service_not_found. Meta's app never sends the action. The rows open
their sheet instead, and the lease code goes.
The VDK's model was the codename the glasses give, greathammerhead for
a Ray-Ban Meta (Gen 2). The names setup already knew by product id move
next to the ids and fill it; a model with none keeps its codename.
# Conflicts:
#	README.md
#	core/assistant/src/main/kotlin/com/macedonga/visor/core/assistant/GlassesEars.kt
#	core/connection/src/main/kotlin/com/macedonga/visor/core/connection/DeviceInfo.kt
…r removed

Scopes go by the client's name, so a socket opened with an old token kept
working, with the new client's scopes after a re-add.
A network client's or web app's name is no package, but it could match an
installed one, which then got read access to the person's captures.
… camera scope

Only Android apps counted, so camera.photo from a network client timed
out. Android apps now count through scopesOf, which checks the certificate.
join runs before the try whose finally leaves: a failed start left the
count high, and a Live an app started never stopped.
…ext one

The other side's close could queue a second loss, which the next round
took as silence: the relay failed after one drop instead of retrying.
The reader could move reads to the new link as the switch gave up, which
then left writes on the old one.
@macedonga
macedonga marked this pull request as ready for review November 9, 2026 14:29
@macedonga
macedonga merged commit 48d8bd9 into main Nov 9, 2026
3 checks passed
@macedonga
macedonga deleted the kit branch November 9, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant